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Abstract 



In this paper, we prove lower bounds on the logical expressibility of optimization prob- 
lems. There is a significant diS^erence between the expressibilities of decision problems and 
optimization problems. This is similar to the difference in computation times for the two 
classes of problems; for example, a 2SAT Horn formula can be satisfied in polynomial time, 
whereas the optimization version in NP-hard. Gradel [E. 91] proved that all polynomially 
solvable decision problems can be expressed as universal (Hi) Horn sentences. We show 
here that, on the other hand, optimization problems defy such a simple characterization, 
by demonstrating that even a simple Hq formula is unable to guarantee polynomial time 
solvability. 

See Section |4] for Duality using Lagrangian Dual to characterize optimality conditions 
— that section also describes using a single call to a "decision machine" (a Turing machine 
that solves decision problems) to obtain optimal solutions. 

1 Background 

The reader is assumed to have some background in Finite Model Theory. If not, the book by 
Ebbinghaus and Flum [EF99j serves as a very good introduction. (See Section [5] for notation 
and definitions.) 

In a recent paper, one of the authors [ ManOSj proved the following: 

Theorem 1. Let A be a structure (instance) defined over a signature a. The value of an 
optimal solution to an instance A of a maximization problem Q can be represented by 



f opt' where x, A, S and rj are defined in TableUl The Horn condition in the formula 
T] applies only to the second order predicates in S, not to first order predicates. 





The converse of Theorem [T] can be stated as follows: 

Proposition 2. If the optimal solution value to a maximization problem Q can be represented 
as in (Oy, then Q belongs to the class P^pt • 

The problem referred to, in Proposition [21 can be cast as an optimization problem as follows: 

Problem 3. Syntactic optimization. 

Given, (i) A structure A, 

(a) a sequence of second order variables S = {^i, 52, • • • , 5p} where each Si is of arity ri 
(1 < i < p). That is, each Si is of the form Si{zi, Z2, ■ ' ' ^ Zr^), where each Zj can take any 
value in the domain of A; and 

(Hi) a quantifier-free conjunction of Horn clauses r/(w,x, S). 

(As in Theorem [71 the Horn condition in the formula r] applies only to the second order 
predicates in S.) 

To Do. For 1 < i < p, assign truth values to each Si, such that the number of tuples w that 
satisfy Vx r7(w,x, S) is maximized. The goal is to achieve the maximum value for optq^A) 
as given in ([ip. 

(We hope the problem definition above is clear. If not, we hope the referees or the editor can 
help us in re-phrasing it.) 

Due to difficulties in computing the optimal solution value for a general maximization problem 
in Popt) Bueno and Manyem |BM08] made the following conjecture: 

Conjecture 4. The optimal value for an instance A of an optimization problem, as mea- 
sured in HP, cannot be computed in polynomial time by a deterministic Turing machine using 
syntactic techniques. We need optimization algorithms that exploit the particular problem 
structure. 

Gate and Stewart |GS08] made an attempt to prove Conjecture [H The decision version of 
MaxHorn2Sat (see Definition [9]) is known to be NP-complete |JS87j . Gate and Stewart were 
able to show a polynomial time reduction from the decision version of MaxHorn2Sat to the 
decision version of Problem [3l thus proving that Problem [3] is NP-hard. 

In other words, the authors in [GSOSj essentially showed that just because the optimal solution 
value to an optimization problem can be expressed in the form in ([T|) does not mean that the 
problem is polynomially solvable, it may also be NP-hard. 

Here we prove a stronger negative result. Notice that the first order part in ([1]) is in Hi Horn 
form (universal Horn). One would expect that if we simplify the expression from Hi Horn to 
Ho Horn (that is, a quantifier- free Horn formula), we should be able to guarantee polynomial 
time solvability. 

Unfortunately this is not the case. We will show below that even a quantifier-free Horn 
expression is unable to guarantee polynomial time solvability. We show this by exhibiting 
such an expression for an NP-hard problem, MaxHorn2Sat defined above. 
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A 


a structure defined over a signature cr; A captures an instance of an 
optimization problem. 


V 


a quantifier- free first order (FO) formula, which is a conjunction of 

llUf fl CldUbco. IXVcLcLli LlldL cl nuill CidU-oC CUllLdlllo dL lliUoL Ullc pUolLlVc 


X 


an m— tuple of FO variables. 


S 


a sequence of predicate symbols or second order (SO) variables; 
S captures a solution to the optimization problem. 


Popt (NPopt) 


P-optimization (NP-optimization) problems. See Definition [S] ([U]) . 


P^pt (NP^^t) 


Polynomially bound P-optimization (NP-optimization) problems. 
See Definition [71 


PNF 


Prenex Normal Form. 



Table 1: Notation 



2 Notation and Definitions 



Definition 5. A P-optimization problem Q is a set Q = {/q, Fq, /q, optq}, where 
(i) Iq is a set of instances to Q, 

(a) Fq{I) is the set of feasible solutions to instance I, 

(Hi) /q(I, S*) is the objective function value to a solution S £ Fq{I) of an instance I £ Iq. 
It is a function f : U/g/q[{-^} ^ ^q(-^)] ~^ (non-negative reals)}], computable in time 
polynomial in the size of the universe of iH, 

(iv) For an instance I £ Iq, optQ(I) is either the minimum or maximum possible value that 
can be obtained for the objective function, taken over all feasible solutions in Fq{I). 

optQ^I) = max ^ fQ{I , S) (for P-maximization problems), 

optQ{I) = min /q(/, 5) (for P-minimization problems), 
SeFQ{i) 

(v) The following decision problem is in the class F: Given an instance I and a non- 
negative constant k, is there a feasible solution S G Fq{I), such that /q(/, S") > k (for a 
P-maximization problem), or /q(/, S) < k (in the case of a P-minimization problem)? 

And finally, 

(vi) An optimal solution Sopt{I) for a given instance I can be computed in time polynomial 
in \I\, where optQ{I) = /q(I, 5opt(/)). (LET ME LEAVE THIS POINT HERE FOR 
THE TIME BEING.) 

The set of all such Y* -optimization problems is the Popt class. 



^Of course, when it comes to computer representation, rational numbers will be used. 

^Strictly speaking, we should use |/| here, where |/| is the length of the representation of /. However, |7| 
is polynomial in the size A of the universe A, hence we can use |/|. 
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A similar definition, for NP-optimization problems, appeared in Panconesi and Ranjan (1993) 
[PR93] : 



Definition 6. An NP-optimization problem is defined as follows. Points (i)-(iv) in Defini- 
tion\^ahove apply to NP-optimization problems, whereas (vi) does not. Point (v) is modified 
as follows: 

(v) The following decision problem is in NP; Given an instance I and a non-negative constant 
k, is there a feasible solution S € Fq{I), such that /q(I, 5) > k (for an NP -maximization 
problem), or fQ{I,S) < k (in the case of an NP -minimization problem) ? 

The set of all such ^Y*- optimization problems is the NPopt class, and Popt ^ NPopt- 

Definition 7. An optimization problem Q is said to be polynomially bound if the value of 
an optimal solution to every instance I of Q is bound by a polynomial in the size of /. In 
other words, for every problem Q, there exists a polynomial pQ, such that 



for every instance I of Q. Pj^pt f^Popt/' ^-^ polynomially-bound P- optimization 

(NF- optimization) problems. Naturally, Popt 

CPopt and NPP^t ^ NPopt. 

Informally, an instance of MaxHorn2Sat consists of a formula in conjunctive normal form 
(CNF), where each clause is Horn, and each clause contains at most two literals. (Such a 
formula is also known as a quadratic Horn formula.) The problem is to maximize the number 
of satisfiable clauses. 

Definition 8. An existential second-order (ESO) Horn expression is of the form ^Sijj, 
where -0 is a first order formula, and S = (5i, • • • Sp) is a sequence of predicate symbols not 
in the vocabulary of ip. The formula ip can be written in Hi form as 



where r] is a conjunction of Horn clauses (rj is, of course, quantifier-free) , and xi {1 < i < k) 
are first order variables. Each clause in rj contains at most one positive occurrence of any of 
the second order predicates Si (1 < i < p). 

A general ESO formula is the same as an ESO Horn expression, except that rj can now be 
any quantifier-free first order formula. 

Problem 9. MaxHorn2Sat. 

Given. A set of clauses a, 1 < i < n. Each clause Ci is one of the following: (i) a Boolean 
variable Xj, (ii) its negation, ^Xj, (Hi) Xj V ^x^, or (iv) ^Xj V ^x/^. 

To Do. Assign truth values to the Xi 's such that the number of satisfied clauses is maximized. 

Definition 10. Iln and Sn formulae. These formulae have n quantifier blocks at the 
beginning, followed by a quantifier-free formula. Each block contains only one type of quanti- 
fier (either existential or universal). Here is the difference between H„ and $]„.■ A n„ C^n) 
formula begins with a universal (existential) block. 




(2) 



ip = 'ixiix2 ■ ■ ■ VxfcT/ = Vx rj. 



(3) 
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MAX Ho is the class of maximization problems whose optimal solution value to an instance 
A of a Problem Q can be represented as 

optQiA) = max [{w : (A, S) ^ r?(w, S)}| (4) 
where rj is a quantifier-free formula. 

3 A Syntactic Expression for MaxHorn2Sat 

We need instances at two different levels. For example, suppose we are given a MaxHorn2Sat 
instance (formula) such as M = (zi V -1^2) A (^3) A {^z^ V -^zi). 

The variables in this instance are Z = {zi, Z2, Z3}, and a structure B maps Z to its universe 
V = {TRUE, FALSE}. 

However, to represent the MaxHorn2Sat instance M as in ([1]) or (jlOp . the variables used will 
be A = {x,y}, and the universe of the structure A would be Z. Diagrammatically, 

X = {x,y} — > Z = {zi,Z2,Z3} — > V = {TRUE, FALSE}. (5) 

A maps (instantiates) X to Z, and B maps (instantiates) Z to V. 

The second order variables S (to be used with A) consists of a single unary predicate S, that 
is S = {S} where S is of arity one. S can be considered as a guess of the map B. In the 
above example, for a certain MaxHorn2Sat clause, if A(x) = zi, A(y) = Z3, S{x) = FALSE 
and S{y) = TRUE, then S would have guessed that B(zi) = FALSE and B{zs) = TRUE. 

If variables x and y appear in a 2-literal MaxHorn2Sat clause, then the clause can assume 
one of the following forms (and represented in the signature of A by the corresponding first 
order predicate on the right): 





BothNeg{x, y) 




FirstNegSecondPos{x,y), or simply FN SP{x,y) 


X V -ly 


FirstPosSecondNeg{x,y), or simply FPSN{x,y) 



If a clause contains only one literal, insert a second literal and set it to FALSE. 
3.1 Counting satisfying clauses 

Our approach is similar to that of Kolaitis-Thakur 1994 |KT94j . where they provide an 
expression for the optimal value for MaxSSat (optimization version). 

We only count satisfying MaxHorn2Sat clauses for the objective function. That is, we count 
the number of tuples (x, y) that satisfy (j), where 

4 

(/) = V '/'i = V </)2 V </.3 V </)4. (6) 

2 = 1 

(The (/>i's are described below.) 
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3.2 Two-literal MaxHorn2Sat clauses 



Two-literal MaxHorn2Sat clauses can be satisfied in one of the following ways: 

= FPSN{x,y) A[S{x) V ^S{y)]. 
^2 = FNSP{x, y) A h5(x) V S{y)]. 
^3 = BothNeg{x,y) A h5(x) V ^S{y)]. 

3.3 One-literal McLx;Horn2Sat clauses 

As mentioned earlier, convert one-literal clauses to two-literal clauses. (We do this, so that 
we can simply count the number of tuples that satisfy cj).) 

If the literal is positive, then create a special predicate called BothPos{x,y), as if the clause 
is X V y; but of course, we set y to FALSE, so y has no effect. So we use 

(pi = BothPos{x,y) A S{x). 

If the literal is negative, then just use 02 above; no need for another Since we set the 
second literal to FALSE, S{y) will never hold. (For (p2 to be true, S{x) must be false.) 

3.4 The complete DNF formula 

For convenience of writing, let us substitute 

A = FPSN{x,y), B = FNSP{x,y), C = BothNeg{x,y), D = BothPos{x,y), 

P = S{x),Q = S{y). 

Then we can rewrite (pi (1 < « < 4) as 

(Pi = {AAP)V{AA^Q), (P2 = {B A^P)V{B AQ), 

03 = (C A -P) V (C A -Q), (1)4 = {DA P). ^ ' 

Prom ([6]), since one of the (pi's should be satisfied for a MaxHorn2Sat clause to be counted 
towards the objective function, 

= 01 V 02 V 03 V 04 = 

(A A P) V (A A ^Q) V (P A -^P) V (P A g) V (C A ^P) V (C A ^Q) V (P> A P). ^ ' 

Write = fci V ^2 V • • • V /c7, corresponding to each of the 7 "conjunct" clauses above in 0. 
That is, A;i = yl A P, A;2 = A ^Q, • • •, and kj = D AP. 
{A A P), ■ ■ ■, {D A P) can be called "conjunct" clauses. 
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3.5 Converting DNF to CNF 



Now (p is in DNF, so we should convert it to CNF. Call the CNF form as ip (or ip{x,y, S), 
to be more accurate). There are 7 clauses in (f) with 2 literals each, so ip will have 2^ = 128 
clauses, with 7 literals each — one literal from each of the 7 clauses in 4>. (See footnotfH) 
From ([1]), we can write ip in lexicographic order as 

-ip = {Ay Ay By By Cy Cy D) ^{Py ^Qy ^py Qy ^py ^Qy P). (9) 

We should ensure that each of the 128 disjunct clauses in is Horn, which is what we do 
next. 

Lemma 11. Each of the 128 clauses in ip is Horn. 

Proof. Note that the literals A, B, C and D are first order (part of the input), hence these 
do not affect the Horn condition; only the P's and Q's and their negations do. 

If there is a 7-literal clause in ip containing literals P and ^P, it can be set to TRUE. Similarly 
for a clause containing Q and ^Q. 

Anyway, we will run into trouble only if we have a clause ipi in -0, that (i) contains literals P 
and Q, and (ii) contains neither -iP nor ^Q. However, can such a clause evaluate to TRUE 
and hence can be "ignored" ? Will such a clause obey the Horn condition? The answer turns 
out to be yes. 

There are only three ways in which we can come across a "P V Q" within a 7-literal clause of 

• Pick P from ki, Q from k4, and one of {A, B, C, D} from the other clauses, to obtain 

ipi = py Ay By Qy cy C'y D. 

This clause of ip contains A, B, C and D — the four types of clauses mentioned in ([7]), 
and one of them must occur; they are mutually disjoint and collectively exhaustive. So 
Ay B y C y D is (always) valid. So ipi can be set to TRUE. 

• Pick P from kj, Q from k^, and one of {A, B, C, D} from the other clauses, to obtain 

tp2 = Ay Ay By Qy Cy Cy P. 

This clause only contains A, B and C, but not D. However, we know that Ay By Cy D 
is valid. If A, B and C are false, then D will be true (the BothPos predicate) — 
this means, every clause in (p is false except kf, which implies that P is true. Hence 
Ay B y C y P is valid, which means ^p2 can be set to TRUE. 

• Pick P from ki and k^, Q from k^, and one of {A, B, C, D} from the other clauses, to 
obtain ip^ = PyAyByQyCyCyP. Apply the same argument as for 'ip2. This 
sets ip3 to TRUE. 

Hence each of the 128 clauses in ip is Horn. □ 
^128 may be "large", but still a finite number. 
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We need one more first order predicate to represent whether a certain (x, y) combination 
actuahy occurs in the MaxHorn2Sat formula, and in which of the four (1 < i < 4) varieties 
it occurs. For each of the four 0j types, introduce a predicate Li{x,y). 

For instance, if L2{z2, z^) is true, it means that {^Z2\/z^) is a clause in the given MaxHorn2Sat 
instance. Similarly, the truth of L3{z2, Z5) implies the existence of the clause {^Z2 V -izs). 

Thus we need to append [Li{x,y)\/ L2{x,y)\/ L3{x,y)\/ L4{x,y)] as the 129th clause. However, 
these four are first order predicates, hence their truth values can be evaluated and substituted. 
This does not affect the Horn condition. 

From all the arguments above including Lemma [TTl we conclude as follows: 

Theorem 12. Let the structure A represent an instance of MaxHorn2Sat defined in Problem 
[21 Then the value of an optimal solution to A can be represented by 

0(A) = max |{(x,y) : (A, 5) ^ (10) 
where tp{x,y,S) is defined in 

Note that ip above is quantifier free (Hq or Sq form). This means 

Corollary to Theorem 1121 and Discussion: Since it is known that MaxHorn2Sat is 
NP-hard, even a Hq Horn expression does not guarantee polynomial time solvability for max- 
imization problems. 

In |Man08j . it was shown that the MaxFlowp^ problem (the MaxFlow problem with unit 
weight edges) cannot be represented in Horn Hq or Horn Si form first order form; it needs a 
Horn Hi sentence. The optimal solution to this problem can be obtained in polynomial time 
using MaxFlow algorithms. 

Hence it is a surprise that while a polynomially solvable problem, MaxFlowp^j has a Horn 
Hi lower bound, an NP-hard problem, MaxHorn2Sat, can be expressed by a quantifier- free 
Horn sentence. 

A similar anamoly was observed by Panconesi and Ranjan (1993) |PR93j : While the class 
MAX Hq (defined below) can express NP-hard problems such as MaxSSat, it is unable to 
express polynomially solvable problems such as Maximum Matching! This suggests that 

Conjecture 13. Quantifier alternation does not provide a precise characterization of com- 
putation time. A hierarchy in quantifier alternation does not translate to one in computation 
time. We need to look at other characteristics of logical formulae such as the number of 
variables, or a combination of these. 

4 Optimality Conditions: Duality to the Rescue 

Recognizing (Verifying) Optimality. In general, the question. Given a solution T to an 
instance A of an optimization problem Q, is it an optimal solution? is as hard to answer as 
determining an optimal solution, necessitating a S2 second order sentence as in ()lip below. 
However, under certain conditions, such as when the duality gap is zero, optimal solutions 
can be recognized more efficiently, and can be expressed in existential second order (ESO, or 
second order Si) logic. 
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Duality Gap is the difference between the optimal solution values for the primal and dual 
problems. For problems such as LP and MaxFlow-MinCut, the duality gap has been shown to 
be zero; that is, they posess the strong duality property. However, for other problems such as 
Integer Programming, there is no known dual problem that guarantees strong duality; hence 
expressions such as (|17|) and (|2Up cannot be derived, at least until a dual that guarantees 
strong duality is discovered. 

The above question can also be phrased as a classical decision problem (for maximization): 
Given a solution T for instance A with solution value f(T), is there another solution S such 
that f(S) > f(T)? 

An optimal solution T to an instance A of an optimization problem Q can easily be repre- 
sented as the best among all feasible solutions S: 

3TVS(/<A[/(A,T) >/(A,S)], (11) 

where (p represents satisfaction of the constraints to A, and / is the objective function referred 
to, in Definitions [5] and [H The formula (p captures the constraints, such as g(x) = b and 
h(x) < c in ^ below. 

[Note that the above formula represents an optimal solution to a maximization problem; we 
can write a similar formula for minimization; simply change the last condition to /(A,T) < 
/(A,S).] 

Recall that a maximization problem Pi in the R'^ Euclidean space can be represented as 
follows |BSS06j : 

Maximize /i(x) : R" ^ R, 
(Pi) subject to g(x) = b, h(x) < c, (12) 
where x G R", b € BJ^^ and c G R'"^ . 



For several optimization problems, an optimal solution can be recognized when a feasible 
solution obeys certain optimality conditions. In such cases, it is unnecessary to represent an 
optimal solution T as in (jlip . The Duality concept in optimization can play an important 
role here. 

Let u G R*"! and v G R™^ be two vectors of variables with v > 0. Given a primal problem 
Pi as in (fT2]) , its Lagrangian dual problem P2 can be represented as (see jBSSOGj ) : 

Minimize 6{u,v) 

(P2) subject to V > 0, (13) 
where 0(u, v) = inf^eR- {/(x) + ET=\ Uig^{^) + EfJi vjh,{x)}. 

Furthermore, 5i(x) = bi [hj{x.) < Cj] is the z*'^ equality [j^^ inequality] constraint. 

We have demonstrated Si second order expressibility using Lagrangian duality in this paper. 
However, other types of duality may be used, such as Fenchel duality or the geometric duality 
or the canonical duality, as long as they provide a zero duality gap, and optimality conditions 
that can be verified efficiently (say, in polynomial time). 
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4.1 Computational Models 

Turing machine (TM) based computational models for solving an optimization problem Q 
come in two flavours: 

Model 1. The input consists of a problem instance such as in (|12|) . If the instance has a 
feasible solution, the output is a string representing an optimal solution; otherwise, the TM 
crashes. Corresponding to classes P and NP in the world of decision problems, the classes 
here are FP and FNP respectively. 

Model 2. In addition to a problem instance such as in (jl2p . the input consists of a parameter 
K, which is a bound on the optimal solution value. The TM is a "decision" machine, that 
is, one whose output is simply a yes or a no; call this machine as Mi. The method then to 
solve Q, by a Turing machine, say M2, is to do a binary search on solution values, calling Mi 
a logarithmic (log number of times, where V is the optimal solution value. Thus we make 
a weakly polynomial^ number of calls to Mi. Each call to Mi involves answering a question 
such as: "Is there a feasible solution S satisfying the constraints, such that the objective 
function value /(A, S) is greater than or equal to KT\ for a maximization problem. 

If the problem answered by Mi is in the NP class, then the complexity of solving Q is P^^, 
since M2 makes a polynomial number of calls to the oracle Mi (and Mi solves a problem in 
NP). 

Similarly, if the problem answered by Mi is in the class P, then the complexity of solving Q 
as , which is simply P (although strictly speaking, this is weakly polynomial due to the 
logy number of calls). 

The method used in Model 2, binary search, has been recognised/adopted for solving op- 
timization problems since the discovery of the class NP. It involves making a polynomial 
number of calls to a "decision TM" (a TM that solves decision problems). 

However, we show in this section that for pairs of problems with a duality gap of zero, a single 
call to a decision TM is sufficient. If the machine answers yes, then the primal and the dual 
problems have optimal solutions; otherwise, neither problem has an optimal solution (one of 
the problems will be infeasible and the other one will have an unbounded optimal solution). 
This is demonstrated by second order Si sentences such as ()17p and ()20p . which implies, as 
per Fagin's result below, that such a machine produces a yes/no answer in NP time. 

Theorem 14. ^FagT^ A decision problem can be logically expressed in ESO if and only if it 
is in NP. 

The following theorem is the deterministic counterpart of Fagin's result. It characterizes P 
as the class of decision problems definable by ESO universal Horn formulae. 

Theorem 15. (Grddel \E. 91^ ) For any ESO Horn expression as defined in Definition\^ the 
corresponding decision problem is a member of P. 

*For a graph problem, an algorithm is strongly polynomial if the running time is a polynomial in the number 
of vertices and/or edges; it becomes weakly polynomial if the running time is a polynomial in the logarithm of 
edge weights. In Linear Programming, this translates to the number of variables/constraints versus the data 
in the coefficient matrix A and the right side vector b. 
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The converse is also true — if a problem V is a member of P, then it can be expressed in 
ESO Horn form — but only if a successor relation is allowed to he included in the vocabulary 
of the first- order formula tp. 

The polynomial time computability in the first part of Theorem 1151 is due to the fact that the 
first order part of formulae representing decision problems can be reduced to propositional 
Horn formulae, which can be solved in time linear in the number of predicates (which are 
unknown second order). 

4.2 Linear Programming 

For example, in the case of Linear Programming (LP), using Lagrangian duality, the primal 
and dual problems P3 and P4 respectively, can be stated as follows: 

(P3) Maximize /i(x) = c-^x, (P4) Minimize /2(y) = b-^y, 

subject to Ax < b, x > 0, subject to A^y > c, y > 0, (14) 

where x, cGR*^, and y,b€R"*, 

after the usual process |Had62j of converting unrestricted variables (if any) to non-negative 
variables, and equality constraints (if any) to inequality constraints, in the primal problem. 
Here, yi (xj) is the i^^ dual (j*'^ primal) variable corresponding to the i*^ primal {j^^ dual) 
constraint. When the primal and dual problems have feasible solutions, then they both have 
optimal solutions x* = {xl,X2, • • • , x^) and y* = {yl,y2, ■ " ^ Um) such that the two objective 
functions are equal: c"^x* = b"^y*. (Almost every book on LP should explain this result. See 
for example, |Had62| .') 

For LP's, the complementary slackness conditions below are known to be necessary and 
sufficient conditions for the existence of an optimal primal solution S and an optimal dual 
solution T: 

ytih- Ai^*) = 0, y*>0, 6i-^,x*>0, i e {1, 2, • • • , m} (15) 
x*(c, -^Jy*) =0, x*>0, c, -Ajy*>0, j G {1, 2, • • • , n} (16) 

where Ai is the i^^ row of A, is the j*^ column of A, (6j — AjX*) = is derived from the 
jth ppiijial constraint, and (cj — Ajy*) = is derived from the j^^ dual constraint. 

Thus the existence of S and T can be expressed as 

3S3T [Vi A [Vj Mj)] A 0p(S) A ^^(T), (17) 

where V'i(^) [^2(j)] logically captures the i''^ [j^^] constraint in (fT5]) [ p^ ] respectively. Also, 
(j)p and model the primal and dual constraints in (|14p respectively. 

We are not concerned about the first order part of the above expression, [Vz 'i/'i(^)] A[Vj "02 (j )] A 
(j^p A (t)d- What is of interest to us is that the existence of optimal solutions for the primal 
and dual problems can be expressed in ESO, existential second order logic; a S2 second order 
sentence as in (llip is unnecessary. 

Applying Theorem 1141 it follows that recognition of an optimal solution, for certain problems 
such as LP, is in the computational class NP. 
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(One could argue that the existence of a feasible solutioqj for an optimization problem, 
satisfying constraints such as Ax < b, x > 0, implies the existence of an optimal solution.) 

4.3 Polynomially Solvable Problems 

But what if the primal and dual problems are polynomially solvable? Can this be reflected 
in expressions such as (IITD ? The answer turns out to be yes. Recall from Theorem (llSp that 
to express polynomial solvability, the first order part of (jl7p needs to be a universal Horn 
formula. 

From our assumption about the polynomial solvability of the primal and the dual problems 
and from Theorem (jlSp . it follows that (pp and 0^ can be expressed as universal Horn formulae. 

As for the complementary slackness conditions ()15p and (I16p . we only need to express 
y*{bi — AiX*) = and x*{cj — ^Jy*) = 0, since the other conditions have been expressed in 
(/)p and (j)d- 

yl{hi — AjX*) = can be expressed as V'i(^) = V B_A{i^X), where Y{i) is a predicate 
which is true iff y* = 0, and B_A{i, X) is a predicate which is true iff bi — AiX* = 0. The 
formula V'l (i) is not Horn. However, since y* = and bi — AiX* = do not occur anywhere 
else in (fT7|) . we can negate the predicates and modify ipi{i). 

As in Theorem [H the Horn condition in the formula r] applies only to the second order 
predicates in S and T. In this case, it applies to predicates that involve unknowns such as 
Xj and yi. 

Let YnotEqO{'i) be true iff y* / 0, and B_AnotEqO{i, X) be true iff bi - AiX* 7^ 0. Using 
these, one can rewrite ipi{i) as 

V'l (i) = -YnotEqQ{i) V -^B_AnotEqO{i, X), (18) 

which is a Horn formula. Similarly, the formula ip2{j) in (|17p can be expressed in Horn form: 

Mj) = ^XnotEqO{j) V ^C^notEqO{j, Y). (19) 

Now that we know that all four subformulae in the first order part of (jl7p can be expressed 
in universal Horn form, we can conclude that the formula in ()17p fully obeys the conditions 
of Theorem list that is, ESO logic with the first order part being a universal Horn formulae 
(that is, the quantifier-free part is a conjunction of Horn clauses). Hence we can state that 

Theorem 16. For a pair of primal and dual Linear Programming problems as in ^14\ ), and 
hence obeying strong duality, the existence of optimal solutions for the primal and the dual 
can be expressed in ESO logic with the first order part being a universal Horn formula, and 
the optimal solutions can be computed in polynomial time (a) syntactically, and (b) by a single 
call to a decision Turing machine (which returns yes /no answers). 

But does strong duality imply polynomial time solvability? This is the subject of another 
manuscript |MS09| . 

word of caution — Feasible solutions, a difference in terminology: Fagin and Gradel [E. 91] have 
syntactically characterized feasible solutions for classes NP and P respectively. However the "feasibility" 
captured by an ESO expression, as described by Fagin and Gradel, also includes an upper (lower) bound on 
the objective function of a minimization (maximization) problem, such as /i(x) > K where K is a. constant 
— not just satisfaction of the constraints such as Ax < b, x > in (|14p . In this paper, we differ from this 
view; when we talk about feasibility, we only refer to satisfaction of constraints such as Ax < b, x > 0. 
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4.4 MaxFlow MinCut 



(Still some work to be done here; The first order part is not universal Horn, but I think it 
should be possible to make it universal Horn.) 

The MaxFlow-MinCut Theorem is another example where Lagrangian duality plays an im- 
portant role in characterizing optimal solutions. The MaxFlow and MinCut problems arc dual 
to each other. At optimality, the values of the two optimal solutions coincide. An optimal 
solution to MaxFlow can be syntactically recognized by an "optimality condition", rather 
than a comparison of the objective function value with those of all other feasible solutions. 

Definition 17. The MaxFlow problem: 

Given. We are given a network G = iy^E) with 2 special vertices s,t E V, E is a set of 

directed edges, and each edge {i,j) G E has a capacity Cij > 0. 

To Do. Determine the maximum amount of flow that can be sent from s to t such that in each 
edge {i,j) € E, the flow f{i,j) is at most its capacity Cij. That is, < f{i,j) < Cij, ^{i,j) G 
E. 

An S-T Cut is a non-empty subset U oiV such that S G U and T & U, where U = V — U . 
[If U is used as a second order predicate, then U{i) is true for all vertices i € U; it follows 
that U{S) is true and U{T) is false;] The capacity of the cut, written as C{U), is the sum of 
the capacities of all edges {i, j) such that z G C/ and j G U: 

C{U) = J2 

{i,j)eE, ieu, j&J 

Definition 18. The MinCut problem: 

Given. Same as the MaxFlow problem. 

To Do. Of all the S — T cuts in G, find a least cut; that is, a cut with the least capacity. 

The optimality condition for the MaxFlow problem is that there exists a least S — T cut, U, 
such that 

• (forward direction) For every edge (i,j) in the edge set E such that i and j G U, 
the flow in (i,j), f{i,j), is equal to its capacity Cij; 

• (backward direction) For every edge (i,j) G E such that i e U and j G U, f(i,j) = 0; 
and 

• The maximum flow, that is, the optimal solution value for the MaxFlow problem, is 
equal to C{U), the capacity of the cut U. 

This condition can be syntactically characterized as 
3U3F yiyj U{S) A ^U{T) 

A [Eii,j) A U{i) A ^U{j) F{i,j, C^j)] (20) 
A [E{i,j) A ^U{i) A U{j) — > F{i,j,0)] A V, where 

U and F are second order predicates; 

E(i,j) is a first order relation which is true whenever is an edge in the input graph; 
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U{i) is true when i G vertex set U; 

F{i,j,v) is true when the flow in the edge equals v; and 
■0 models the flow conservation constraint at all nodes. 

Once more, with the help of previously proven optimality conditions, we have been able to 
characterize the primal optimal solution F and the dual optimal solution U, in existential 
second order logic (ESO). 

Similarly in Convex Programming, the Karush-Kuhn- Tucker conditions provide sufficient 
conditions for the optimality of a feasible solution. 

5 Zero Duality Gap 

Note. Here, we only express the existence of optimal solutions. We do not compute opti- 
mal solutions (we do not provide a method to compute them). Gradel [E. 91] provided an 
expression for the existence of a feasible solution for polynomially solvable problems. What 
we provide here is an improvement on his result, for problems that obey strong duality. 

We observe that expressions such as those in ()17p and (|20p are possible only if there is no 
duality gap, that is, when the duality gap is zero. The primal optimality condition implies 
dual feasibility and vice versa. 

(ARE THE FORMULAE IN ([IT]) and ([20]) Horn? If so, using the "propositional Horn formula 
in linear time" property, can you get something algorithmic out of it?) 

To our knowledge, all known problem-pairs with a zero duality gap (also known as strong 
duality) are polynomially solvable. The decision versions of all such optimization problems 
can be shown to be in the complexity class NP n CoNP |MS09j . 

Problems in NP n CoNP can be expressed in both ESO and USO (universal second order 
logic), since USO precisely characterizes problems in USO. 

Does strong duality imply polynomial time solvability? This is the subject of another manuscript 
|MSn9j . 

(How does aU this relate to the POLYNOMIAL HIERARCHY? Saddle Points? (Linear) 
complementarity problems?) 
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